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ABSTRACT MDRC, an independent, nonprofit research firm, partnered with New Visions for Public 
Schools, which supports a network of district-run high schools in New York City, to de- 
sign and evaluate an intervention aimed at improving high school students’ attendance. 
The intervention used text messaging to send parents daily absence updates and week- 
ly attendance summaries; students were randomly assigned to have their guardians re- 
ceive messages. The evaluation found that the intervention did not change attendance 
rates in the second semester of the 2015-2016 school year. 


INTRODUCTION 

Graduating high school is a prerequisite for economic success, and yet nationwide barely 80 
percent of students in public schools graduate on time. Reforms to address this problem range 
from restructuring schools to revamping curricula. However, a growing body of research 
suggests that a low-cost way to boost graduation might begin with simpler measures: getting 
students to show up for school. 1 

Currently, far too many students do not, especially in urban high schools. Nearly one in five 
high school students are chronically absent each year, meaning they miss 15 or more days 
of school. 2 Even if those absences are not consecutive, in terms of lost instructional time, 
missing that amount of high school is the equivalent of missing entire eras of global history or 
never reading seminal pieces of literature. 
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As students become more independent in high school, 
parents may not know whether their children make it 
to school. 3 In New York City, parents’ information gap 
is even wider because of the structure of the city- 
wide high school choice system: Tens of thousands of 
students travel across five boroughs to attend school, 
making it challenging for parents to track whether 
their children arrive. 

The high rates of absence mentioned above suggest that 
the existing methods schools and districts are using to 
inform parents when their children are absent are not 
effective enough. Many districts send parents auto- 
mated voice messages (“robocalls”) alerting them to a 
student’s absence. Attendance Works, a national non- 
profit organization monitoring attendance patterns and 
research into attendance, notes that calls from a princi- 
pal can be useful for occasional awareness messages but 
that regular alerts in a robotic voice are often ignored 
by parents, especially in an age when parents tend to be 
overburdened by information. 4 Staff members in vari- 
ous school districts around the country have acknowl- 
edged the same to MDRC in personal communications. 
Moreover, parents may view robocalls from schools as 
discouraging at best and overly punitive at worst. But 
few districts are willing to stop this form of outreach 
without a clear replacement, and there is as yet little 
evidence regarding such replacements. 

Together, New Visions for Public Schools and MDRC 
designed and evaluated one such potential replacement 
in New York City: a low-cost text-messaging interven- 
tion. New Visions is a school-support organization pro- 
viding data tools, professional development, coaching, 
and other forms of assistance to 70 district-run schools 
that enroll mostly low-income students (in addition 
to its own charter schools and other New York City 
Department of Education community schools). MDRC 
is a nonprofit, nonpartisan research organization with 
more than 40 years of history designing promising 
new interventions, evaluating existing programs using 
rigorous research designs, and providing technical 


assistance to build better programs and deliver effective 
interventions on a large scale. 

THE INTERVENTION 

The goal of this demonstration was to develop and test 
an intervention that would catch parents’ attention and 
provide them with frequent and accurate information 
about their students’ absences from school, without 
burdening school staff members or using other valuable 
school resources. 

Mode 

Text messaging was selected as a viable mode for 
delivering information for several reasons. First, there 
is emerging evidence that text messaging can promote 
positive routines among students and parents. 5 Second, 
text messaging is hypothesized to be more noticeable 
than automated phone calls because parents can view 
the messages easily and quickly, and it is hypothesized 
to be more effective because text messages can seem 
warmer than a robotic voice. And third, a text-messag- 
ing approach can be easily expanded to serve thou- 
sands of people at negligible additional cost, which a 
staff-driven approach cannot. 

New Visions therefore wrote software that accessed 
daily attendance data and triggered automated text 
messages when students were absent. While there have 
been other randomized controlled trials of outreach 
to parents about attendance, this study is unusual in 
that it was able to test same-day absence informa- 
tion conveyed in text messages. The handful of other 
randomized controlled trials that used text messaging 
did not evaluate how attendance changed in response 
to regular information over time. These trials found 
increases in attendance in the day or week following a 
text message, but they did not examine whether pro- 
viding frequent absence information boosts attendance 
throughout a school semester or year. 6 
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Content 

In designing the text-messaging intervention, the study 
team drew on three mechanisms shown to be effective 
in recent experiments: 

1. GIVING PARENTS THE CHOICE TO OPT OUT RATHER 
THAN ASKING THEM TO OPT IN. The intervention 
offered messages to all parents, with the option 
to opt out. Setting the “default” to send messages 
rather than asking parents to sign up simplifies 
parents’ access to information and reduces hassle. 7 

2. REMINDERS. Parents of absent students received 
same-day text message updates and all parents in 
the intervention group received weekly attendance 
summaries. 

3. PERSONALIZATION. Messages were sent in the 
family’s home language and showed the student’s 
first name and the school’s name, as well as the rel- 
evant school phone number to call if a parent had 
questions. 8 A given student could have multiple 
guardians, each receiving messages in a different 
language. 

Because this test of the messaging system was its first, 
the research team opted for a strategy that focused sim- 
ply on providing information neutrally. The messages 
did not frame that information in positive or negative 
terms or prompt parents to take any particular actions. 

Timing 

The intervention focused on the second semester of the 
school year, when frequent holidays interrupt school 
routines and student attendance tends to plummet. 
Daily messages were sent between 6 p.m. and 10 p.m. 
on the day a student was absent, and weekly summary 
messages were sent on Saturday mornings. 


Participation 

The goal of this demonstration was to implement the 
intervention as a district would in practice. Doing so 
meant delivering the intervention to parents of all high 
school students randomly assigned to receive it. This 
approach also offered benefits to the evaluation: It is 
difficult to predict for which students the intervention 
would be most effective. Targeting any one subgroup 
could mean missing potential effects on other sub- 
groups or even overall effects on the full population of 
students. 

In addition, to maximize the amount of information 
conveyed about each student, the intervention was 
sent to all guardians associated with a student who 
had valid cell phone numbers (rather than just one 
guardian). 9 

Relationship to Existing Efforts to 
Improve Attendance 

The intervention was meant to enhance “business-as- 
usual” outreach, not to replace it. Text messages did not 
supplant robocalls or other forms of communication 
that schools may have initiated. 10 Rather, the text mes- 
sages provided extra information in a new form, which 
could have helped if parents were not noticing or pay- 
ing attention to information received in other forms. 

THE STUDY 

As mentioned earlier, the study took place in New York 
City district-run public high schools belonging to the 
network managed by New Visions. To be eligible for 
the study, schools had to be able to upload attendance 
data daily, with at least 8o percent accuracy in record- 
ing students’ absences. 11 To create the greatest possible 
contrast between the group receiving the intervention 
and the control group, the study excluded a few schools 
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that were already using text messaging to communicate 
with parents about attendance. The study then ran- 
domly assigned high school students and their guard- 
ians to receive the text-message intervention described 
above or to receive usual forms of outreach (which 
could have varied from school to school). 12 

The Sample 

The analysis sample consists of high school students 
in n schools who had at least one parent or guardian 
residing with them who (1) had active cell phones at the 
start of the study, (2) were authorized to be text mes- 
saged, and (3) did not opt out before students were ran- 
domly assigned. 13 Using these criteria, 3,957 students 
were selected for the study. Most students had just one 
guardian receiving messages, but at least 29 percent had 
multiple guardians. Generally, the text-message partic- 
ipants and those receiving usual outreach were similar 
to each other at the start of the intervention in ways 
relevant to the study. They had similar first-semester 
attendance rates (both about 90 percent), proportions 
of chronically absent students (both about 23 percent), 
and proportions with a non-English speaker in the 
household (both about 25 percent). 

The study results apply to those students whose parents 
have active cell phones. Because of the additional sam- 
ple eligibility criteria described above, the study sample 
does not represent all of the students in the study 
schools: 81 percent of students whose guardians had 
active cell phones as of January 2016 are in the analy- 
sis sample; the other 19 percent are not in the sample 
because they had guardians who were unauthorized or 
who opted out before random assignment occurred. 14 

Analysis and Results 

The primary outcome of interest was each student’s 
attendance rate during the time that student was 
enrolled in the 71 days of the intervention (from 


February break until the last day of classes before 
state exams). This definition retains all students in the 
sample, including students who transferred or dropped 
out. All students were weighted equally. 15 The study 
estimated the average effect of intended participation 
in the New Visions text-messaging intervention, rather 
than the effect of students’ guardians actually receiving 
text messages. 16 This “intent-to-treat” effect captures 
the reality that any school district would face when 
implementing a system like the one in this evaluation: 
Some parents will change phone numbers or opt out 
after an outreach effort begins. 

The study did not find that the text messages had a 
statistically significant or meaningful effect on stu- 
dent attendance. Both the text-message and control 
groups have second-semester attendance rates of about 
86 percent. The estimated effect is small in practical 
terms (less than 1 percentage point, or about half a day 
of additional attendance, on average) and in terms of 
statistical significance (it is not significant at the 0.10 
level typically used in education research). The study 
has sufficient sample size to draw a conclusion about 
the true effect of the intervention. 17 

As shown in Figure 1, second-semester attendance 
follows the same pattern for both the text-message 
group and the control group: Attendance declines over 
the course of the semester, with the steepest decline in 
June. It typically declines after a holiday and after each 
weekend, with a bit of an increase midweek. Although 
the text-message group appears to attend at a slightly 
higher rate at times during the trial, there is no con- 
sistent pattern suggesting that that group attended at a 
higher rate during some months or on certain days of 
the week. 

Consider also that there is a strong positive correlation 
(0.9) between the first- and second-semester attendance 
rates for both the text-message group and the control 
group, and that attendance declines for both groups in 
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FIGURE 1 

Daily Attendance Rates of Students in the Text-Message Group and the Control Group 
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SOURCE: Daily attendance data from New Visions for Public Schools and the New York City Department of 
Education. 


NOTE: Dark shading indicates holidays and breaks in the lines indicate nonschool days (generally weekends). 
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the second semester. The intervention did not shift the 
attendance trajectory or patterns that the text-message 
group set in the first semester. In fact, similar propor- 
tions of students in the two groups improved their 
attendance between semesters (30 percent in the con- 
trol group compared with 29 percent in the text-mes- 
sage group) and maintained the same attendance rate 
(rr percent, compared with ro percent of the control 
group). 

The study conducted exploratory analyses that inves- 
tigated effects within particular types or subgroups of 
students. There were no statistically significant differ- 
ences in impacts on students in different grade levels; 
on students with English-speaking parents compared 
with those with at least one non-English-speaking 
parent; or on those with one parent receiving text mes- 
sages compared with those with two or more. 

The study also conducted an exploratory analysis that 
investigated effects on the rate of chronic absenteeism 
(as mentioned above, defined as missing r5 or more 
days of school in a year). That outcome can sometimes 
show gains even when the overall attendance rate does 
not. This evaluation did not find an impact on chronic 
absenteeism either, however. 

DISCUSSION 

The design of the text-messaging intervention was 
based on the hypothesis that parents and guardians 
of high school students lack consistent and accurate 
information about whether their children make it to 
school, and that current school-outreach methods or 
district-wide efforts such as robocalls are inconsistent 
or not noticed. The intervention aimed to provide 
consistent and timely information that could be more 
noticeable. There are several possible explanations for 
the lack of meaningful effects detected: 


1. INFORMATION (KNOWLEDGE ABOUT ABSENCES) 
MAY NOT SUFFICE. 

The problem may be that guardians lack resources or 
time to address students’ absences, and therefore need 
encouragement, support, or even referrals to services 
that can help them resolve those challenges. Alter- 
nately, guardians may need to receive information 
worded in ways that prompt them to action. 

Other attendance interventions have focused on cor- 
recting parents’ beliefs, providing information about 
a student in comparison with others, and mentoring. 
On the lower-intensity end of the spectrum, an exper- 
imental evaluation by Rogers and Feller found that 
occasional summaries of attendance information sent 
by postcard boosted attendance in an urban district. 
Those postcards were designed to correct parents’ 
incorrect beliefs about students’ absences. 18 In another 
study, reminders before a holiday or another day 
that had historically seen many absences increased 
attendance by 2 percentage points on the targeted 
day, though the study in question did not report on 
sustained effects. 19 On the higher-intensity end of 
attendance interventions, a nonexperimental evalua- 
tion found that attendance increased by 5 percentage 
points when New York City high school students with 
histories of chronic absenteeism were assigned mentors 
to provide attendance encouragement and help them 
address the underlying reasons for their absences. 20 

2. THE TARGET POPULATION MAY HAVE BEEN TOO 
BROAD TO DETECT EFFECTS. 

The theory of change for this intervention was that 
reaching guardians of all students, rather than target- 
ing chronically absent students, would boost average 
attendance rates. However, if one wants to increase 
the attendance rates of students who are chronically 
absent, and shift the overall distribution of students’ 
attendance rates higher, then practitioners may need 
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a more intensive or longer-running intervention. The 
average second-semester attendance rate among those 
who opted out during the study was about ro percent- 
age points lower than the average attendance rate of 
those who remained. If lower-attending families could 
have benefited more from an intervention, perhaps 
they needed referrals to services or something more 
than information to remain in the study. An informa- 
tion-only intervention may not be a solution for the 
guardians of chronically absent students. 

3. THE INTERVENTION MAY NEED MORE INTENSITY OR 
DIFFERENT TIMING. 

A second-semester launch assumed that it is possible to 
shift attendance trajectories midway through the year 
by providing information during the time of year when 
attendance starts to wane. However, to shift or reverse a 
pattern that has been set early in the year may require a 
more intensive intervention than what was offered. The 
intervention might have to last longer, have different 
content, or engage parents and students to a greater 
extent. 

4. THERE MAY NOT HAVE BEEN SUFFICIENT CONTRAST 
BETWEEN THE INTERVENTION AND “BUSINESS- 
AS-USUAL” EFFORTS TO INFORM PARENTS ABOUT 
ATTENDANCE ISSUES. 

Midyear interventions that provide only absence 
updates may not represent enough of a contrast with 
business-as-usual notification to guardians to produce 
a different result. Schools may have been conducting ad 
hoc outreach to both text-message and control group 
students, since they did not know which students or 
guardians were sent text messages. Such ad hoc efforts 
could have reduced the contrast between the text- 
message group and the control group. It also is possible 
that student guardians were more familiar with the 
business-as-usual robocalls and therefore noticed them 
more, or that voice mails plus text messages either did 


not present enough extra information to be useful or 
actually presented too much, and overloaded them . 21 

Even low-cost interventions need to be substantial 
enough to warrant the effort it takes to implement 
them. The program contrast created by purely infor- 
mational text messages appears to be inconsequen- 
tial, when provided on top of other information and 
outreach. However, if this intervention were intro- 
duced in schools where no other attendance outreach 
was happening, then it might represent a meaningful 
enhancement. 

RECOMMENDATIONS 

The research partnership identified some potential 
directions that districts and schools interested in test- 
ing attendance-messaging interventions could pursue: 

• TARGET NINTH-GRADE STUDENTS IN THEIR FIRST 
SEMESTERS: Given the strong relationship between 
first- and second-semester attendance, it may be 
useful to design an intervention that attempts to 
affect students’ attendance patterns as they make the 
transition to high school. 

• BUILD MORE COMPREHENSIVE RELATIONSHIPS WITH 
PARENTS: It may be that parents who only received 
messages about attendance saw them as punitive. 
Text messaging may be useful if it builds trust 

and connection with parents. It could be useful to 
develop and test a more comprehensive text-message 
outreach plan that provides more than just one- 
dimensional messages and alerts. 

• FOCUS ON PARENTS AND STUDENTS TOGETHER: 

While this study focused on parents as a factor in the 
attendance equation, attendance in high school is a 
decision largely made by students and supported or 
monitored by parents and schools. It may be helpful 
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to engage parents and students together to identify 
and address students’ barriers to attendance. 

NOTES 

1 Allensworth and Easton (2007); Bruce, Bridgeland, Fox, 
and Balfanz (2014); Balfanz and Byrnes (2014). 

2 U.S. Department of Education, Civil Rights Data Collec- 
tion (2016). 

3 Attendance Works (2016). 

4 Flammond (2014). 

5 Castleman and Page (2015); York and Loeb (2015); 
Bergman (2016). 

6 Rogers and Feller (2016b). 

7 Rogers and Feller (2016a); Bergman and Rogers (2016). 

8 Castleman and Page (2015). 

9 The intervention used a Google Apps script in conjunc- 
tion with Twilio, which supports large-scale text messag- 
ing, to send messages automatically to relevant parents 
and guardians. Parents could reply at any time during the 
study to opt out or to request messages in a different 
language. The script automatically updated with parents’ 
preferences and removed students who became inactive, 
whose parents had opted out, or whose parents’ con- 
tact information had changed. A list of absent students 
submitted by schools to the Department of Education 
was accessed by a New Visions data warehouse by 1 
p.m. each day. The script then limited the list of absent 
students to those in the text-message group. For each 
student who was absent that day, the script would gener- 
ate the prespecified content of the text message in the 
preferred language, and load those messages and corre- 
sponding cell phone numbers into a spreadsheet. Twilio 
would then access the sheet each afternoon and send 
messages to guardians of students absent on that day. 

10 New Visions provides an array of attendance-oriented 
tools such as an attendance heat map, which color-codes 
student attendance rates and updates itself automatically 
throughout the year. 

11 This 80 percent criterion meant there was some error 


(both false positives and false negatives) in daily absence 
updates. The error proved to be too great at one school, 
which withdrew from the study in the first month due to 
concerns that the absence notifications were not accurate 
for all students. The study went from 45 to 41 random 
assignment blocks as a result. 

12 Students were randomly assigned to the text-message 
group or the control group in equal proportion, within 
blocks defined by grade levels within schools. 

A potential threat to the validity of any student-level 
randomized controlled trial is that students in the control 
group could actually receive the intervention for some 
reason, a problem referred to as “crossover.” Because 
New Visions owned the text-messaging system, and 
because New Visions rather than the schools controlled 
that system, crossover was nonexistent. It is theoretically 
possible that there could have been “spillover” — that one 
student’s assignment to the intervention group could have 
affected another. For example, because the study ran- 
domized students regardless of their households, rather 
than designating a single “focal" student within each 
household, it is possible that students in the control group 
could have had guardians who received text messages for 
siblings in the text message group. Such a scenario theo- 
retically could have made the control group act more like 
the text-message group. In the analysis sample, however, 
only 100 students in the control group shared guardians 
with students in the text-message group. 

The analysis plan was prespecified and registered in the 
spring of 2016 with the What Works Clearinghouse Ran- 
domized Controlled Trial Registry, which has subsequently 
stopped posting plans. 

13 The sample did not include foster parents and guard- 
ians of homeless students, but did include guardians who 
were grandparents. 

14 There is always some degree of self-selection among 
those who opt out. Despite some baseline differences 
between those who remained in the study and those who 
left or were excluded, one can still draw conclusions about 
the effect of messaging parents. 

The sample can be classified and compared in several 
ways. Students in the study had a 90 percent average 
first-semester attendance rate, compared with an 88 per- 
cent average first-semester attendance rate among those 
who opted out before the study began. Students whose 
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guardians remained in the study had a 91 percent average 
first-semester attendance rate, compared with an 84 per- 
cent average first-semester attendance rate among those 
whose guardians opted out during the study. Although 175 
of the students assigned to the intervention group had 
all of their guardians opt out of receiving text messages 
after random assignment, these 175 students remained 
in the analysis sample. Guardians who opted out, however, 
received text messages only before they opted out. Also, 
during the study, 61 of 1,976 text-message-group stu- 
dents became inactive (including students who dropped 
out and those who transferred to other schools). These 
students also remained in the analysis sample. 

15 The analysis did not weight students based on the 
amount of time they were enrolled because it was import- 
ant not to underrepresent students who transferred or 
dropped out. In general, such students are more likely to 
need attendance support. Moreover, effects on atten- 
dance could be confounded with effects on dropping out. 

The analysis accounts for different schools and grades 
having different numbers of students and gives more 
weight to those school-grade blocks with more students. 

16 Estimation of the average intent-to-treat effect 
accounted for the blocked randomization with indicator 
variables for each school-grade block. In order to improve 
the precision of the estimate, baseline measures of 
student age, gender, and first-semester attendance rate 
were also included in the estimation model. There were no 
missing values for baseline measures for those students 
in the analytic sample. 

17 The study also found no effect on the number of 
days of consecutive absence. Statistical power based 
on sample size helps determine the smallest effect that 
can be detected. With 41 randomization blocks and an 
average (based on the harmonic mean) of 69 students per 
grade-level block randomly assigned to the intervention 
with a probability of 50 percent, the estimated minimum 
detectable effect size with a two-tailed test (with the 
typical 80 percent power and 0.05 statistical significance 
level) was 0.017 standard deviations. To express this 
effect in percentage points or days, the analysis uses the 
standard deviation of the outcome, which was 0.22 per- 
centage points, to conclude that the study could detect an 
effect as small as a 0.4 percentage point increase or 0.3 


additional days of attendance. The explanatory power of 
baseline measures and block indicators (R2) was 0.97. 

18 Rogers and Feller (2016a). 

19 Rogers and Feller (2016b). 

20 Balfanzand Byrnes (2014). 

21 At the conclusion of the study, the intervention team 
sent guardians who remained in the study a single survey 
question via text message asking them their opinion of the 
intervention; about one-third responded. The vast majority 
of those who responded said they liked receiving infor- 
mation by text message, but with such a low response 
rate one cannot be sure that they represent parents who 
did not respond. Comparing observed characteristics, 
respondents’ children had somewhat higher first-semester 
attendance rates than nonrespondents’ children (a 95 
percent average attendance rate versus 90 percent), and 
a lower proportion of respondents than nonrespondents 
spoke a language other than English (23 percent versus 
28 percent). 
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